Freehand input information management device and method

ABSTRACT

A device for storing information electronically and which has a pen-sensitive screen enabling a user to make freehand input has the screen divided into two areas--a main area and a subsidiary area adjacent the main area. 
     The user can tag items of freehand input causing a tag item to be displayed alongside the freehand input and can subsequently manipulate (eg delete, move, copy) the freehand input by selecting and manipulating the associated tag item. 
     Tag items provide a permanent way of structuring data and are an elegant way of uniting several data manipulations.

TECHNICAL FIELD

The present invention relates to a device for storing informationelectronically and which has a pen-sensitive screen for a user to makefreehand input. The device may be a handheld device for the storage andmanipulation of personal data with a pen/stylus for user input.

BACKGROUND ART

There are already several pen-based handheld computer products on themarket for the storage and manipulation of personal data. Examples ofthese are the Apple Newton, the Tandy Zoomer and the Amstrad PenPad. Inthese products the user has to become familiar with several unrelatedmechanisms in order to perform manipulations on the data stored in thedevice. The term `manipulations` refers to operations such as moving,deleting, copying etc of data. In addition, the methods of selectingdata for manipulation usually effect only a transient selection so thatfurther manipulation of the same data requires the data to bereselected.

DISCLOSURE OF INVENTION

According to the present invention we provide a computer having:

means enabling a user to make freehand input;

means for displaying the freehand input in a main screen area;

means for allowing the user to tag an item of freehand input so that atag item is subsequently displayed adjacent the said freehand input in asubsidiary screen area associated with the main screen area;

whereby the tagged freehand input can subsequently be manipulated bymeans of the tag item.

The present invention has the advantage of providing a simple andelegant way of uniting several manipulations using the single artefactof tag items. Tag items provide a way of permanently structuring dataand can be reused as often as desired. The artefact of tag items isparticularly useful in the context of a small screen device for storingand manipulation of personal information where convenient datamanipulation is particularly important.

In the embodiment to be described, a tag item is selected prior tomanipulation of the tagged freehand input and creation of a tag itemautomatically results in selection of the newly created tag item.

In a particular embodiment to be described, the device is configured sothat only the subsidiary screen area recognises user input relating totag item manipulations. The device may comprise means enabling a user toselect a plurality of tag items at any one time. This complicates theuser interface of the device but may be useful in certain application,for example in associating more than one address with a message.

The tag item manipulations typically comprise moving, copying anddeleting tag items and the associated freehand input and may furtherinclude the creation of a new page with the tagged item as its title.

BRIEF DESCRIPTION OF DRAWINGS

A particular embodiment of the present invention will now be described,by way of example, with reference to the accompanying drawings of which:

FIG. 1 is a diagram of a device according to the present invention;

FIG. 2 is a diagram of the screen of the device of FIG. 1.

BEST MODE FOR CARRYING OUT THE INVENTION AND INDUSTRIAL APPLICABILITY

FIG. 1 shows a pen-based handheld device 10 for storing and displayingpages of electronic information comprising a display screen 12. The usermakes input using a pen (not shown). The device 10 also comprisesbuttons 13,14,15 and 16 for copying, moving (`tear` and `stick` buttons)and deleting items. The device 10 also comprises other buttons which arenot relevant to the present invention.

Referring to FIG. 2, the screen 12 is divided into two parts--a mainpart 17 and a subsidiary screen area 18.

The user can make freehand or `scribble` input using the pen on thesurface of the screen 12. An example is shown in FIG. 2 in which thewords `old` and `new` have been written by the user. It is possible totag any such freehand input with tag items, examples of which arereferenced 20 and 22. In order to create a tag item against an item offreehand input, the user simply taps the pen in the area 18 adjacent tothe relevant freehand input. For example, tapping the pen in the area 18adjacent to the word `old` in FIG. 2 causes the tag item 20 to appear.

The following algorithm provides a simple example of how pen strokes maybe associated with tag items (`bullets`):

    ______________________________________    AssociateStrokeWithBullet(stroke){    bullet = GetTopBulletOnPage();    if((bullet == NO.sub.-- BULLET) OR    (topOf(bullet) BELOW topOf(stroke))){    AssociateStrokeWithPage(stroke);    }else{    do{    previousBullet = bullet    bullet = GetBulletBelow(bullet);    }until((bullet == NO.sub.-- BULLET) OR (topOf(bullet) BELOW    topOf(stroke)));    AssociateStrokeWithBullet(previousBullet);    }    ______________________________________

In this case a stroke is associated with the nearest tag item whose toppoint is above or level with the top point of the received stroke. If nosuch tag item exists then the scribble is associated with the pageinstead (this simply reflects the fact that the stroke cannot currentlybe selected using any tag items).

This algorithm will be called when a new stroke is input by the user.The algorithm also needs to be used whenever tag items are moved orcreated. In this case it is necessary to reevaluate the association ofevery scribble on the current page. There are many possible improvementsfor this algorithm. For example, the algorithm may be biassed in favourof associating a new stroke with the same tag item as the previousstroke. It may also use information about the bottom and mid-height ofthe stroke and the tag item. Optimisations for faster computation arealso possible.

A tap with the pen on an existing tag item causes that tag item to beselected. Selection of a tag item is indicated by a colour change asshown for the tag item 22 in FIG. 2. Only one tag item can be selectedat any one time. The next step is for the user to indicate whatmanipulation it is desired to make on the selected tag item andassociated freehand input. This is done using the buttons 13,14,15 and16 of the device 10.

For example, if it is desired to move the selected tag item 22, the userpresses the `tear` button 14 and the selected item drops to the bottomof the page awaiting the user to choose where to move the item to and topress the `stick` button 15 which completes the manipulation. If it isdesired to copy the selected item, the user presses the copy button 13and the selected item drops to the bottom of the page awaiting the userto choose where to copy the item to and to press the `stick` button 15which completes the copy manipulation. If it is desired to delete theselected tag item, the user presses the button 16 which causes the tagitem 22 and the text `new` to be deleted.

A double tap by the pen on a tag item causes the freehand inputassociated with the tapped tag item to become the title of a new page.In this way a hierarchy of pages and sub-pages can be constructed. Thetitle of the new page is tagged with a reverse arrow which, when tappedreverts the display to the original page. The operation is recursive inthat it is possible to create tagged freehand input in the sub-page andto double tap on the tag to create a further sub-page with the latestfreehand input as its title. Likewise the new sub-page title will have areverse arrow in the subsidiary screen area to allow the user to moveupwardly in the hierarchy of sub-pages.

A pseudo-code implementation of this embodiment of the present inventionis as follows. The operations described below show how tag items (called`bullets`) may be selected or deselected and the operations that areaffected by there being a selected tag item.

    ______________________________________    PenDown(position){    if(InMarginArea(position)){    bulletUnderPen = GetAnyBulletUnderPen();    if(bulletUnderPen){    if(DoubleTap()) ShowBulletAsSubPage(bulletUnderPen);    else{/* Toggle bullet state */    if(SelectedBulletIs(bulletUnderPen))    SetSelectedBullet(NONE);    else SetSelectedBullet(bulletUnderPen);    }else{    SetSelectedBullet(CreateANewBullet());    }    }else{/* Not in margin */    SetSelectedBullet(NONE);    }    }    PenUp(position){    if(NOT SelectedBulletIs(NONE)){    /* If some bullet is selected at this point . . .*/    EraseCurrentPenStroke();    /* . . then this is not an ordinary pen stroke . . . */    MoveSelectedBulletAnd ScribbleTo(position);    /* . . . it's a bullet move. */    }    }    UserHasPressedDelete(){    if(SelectedBulletIs(NONE)) DeleteCurrentPage();    /* if no selected bullet, delete page */    else DeleteSelectedBulletAndScribble();    }    UserHasPressedTear(){    if(SelectedBulletIs(NONE)) TearCurrentPage();    else TearSelectedBulletAndScribble();    }    UserHasPressedCopy(){    if(SelectedBulletIs(NONE)) CopyCurrentPage();    else CopySelectedBulletAndScribble();    }    ______________________________________

The invention provides a simple and user-friendly way of manipulatingfreehand input. There are several possible modifications and variationson the above-described embodiment. The range of manipulations unified bytag items may vary according to the particular device embodying thepresent invention. An example would be that selection of a tag itemassociated with a telephone number would cause the number to be rungwhen a `send` or `phone` button is pressed.

The appearance of the tag items can be varied according to requirementsand may be made customisable by the user. Different types of tag itemcould be used for different types of data eg. a sub-page which hasfurther data at a lower level could be tagged with a differentlypresented tag item from sub-pages without such further data.

The subsidiary area 18 shown as a left hand margin in FIG. 2 could madelarger or smaller and could appear on any edge of the displayed page.

The device 10 may be configured so that if tag items are moved withinthe area 18, automatic rearrangement of the tag items occurs to avoidone tag item overlapping with another or freehand input associated withone tag item overlapping with freehand input associated with another tagitem. This can be achieved by causing the original tag item to move downthe screen a predetermined distance to provide enough space for theother tag item and associated input.

Another possible feature would be to allow more than one tag item to beselected at any one time. An example of when this feature could beuseful is when the tag items are associated with addresses and it isdesired to send a message to more than one addressee. Since theselection of more than one tag item could lead to ambiguity with respectto some manipulations eg move, this feature would probably best be onewhich the user could invoke as required.

Tag items may be able to be created in ways other than by a tap of thepen in the area 18. For example, a tag item may be created whenever theuser performs a pen stroke which is entirely contained in the area 18.The tag item would be positioned according to the position of apredetermined part of the stroke eg the beginning or the end of thestroke.

It will be seen that the present invention provides a particularlyeffective way of marking and manipulating data stored on small-screendevices.

I claim:
 1. A device for storing information electronicallycomprising:means enabling a user to make freehand input; means fordisplaying the freehand input in a main screen area; means for allowingthe user to tag an item of freehand input so that a tag item issubsequently displayed adjacent the said freehand input in a subsidiaryscreen area associated with the main screen area; whereby the taggedfreehand input can subsequently be manipulated by means of the tag item.2. A device according to claim 1 wherein a tag item is selected prior tomanipulation of the tagged freehand input.
 3. A device according toclaim 2 wherein creation of a tag item automatically results inselection of the newly created tag item.
 4. A device according to claim3 which is configured so that only the subsidiary screen area recognizesuser input relating to tag manipulation.
 5. A device according to claim3 wherein tag item manipulations comprise moving, copying and deleting.6. A device according to claim 2 comprising means enabling a user toselect a plurality of tag items at any one time.
 7. A device accordingto claim 6 wherein tag item manipulations comprise moving, copying anddeleting.
 8. A device according to claim 2 which is configured so thatonly the subsidiary screen area recognizes user input relating to tagmanipulation.
 9. A device according to claim 2 wherein tag itemmanipulations comprise moving, copying and deleting.
 10. A deviceaccording to claim 1 which is configured so that only the subsidiaryscreen area recognises user input relating to tag manipulation.
 11. Adevice according to claim 10 wherein tag item manipulations comprisemoving, copying and deleting.
 12. A device according to claim 1 whereintag item manipulations comprise moving, copying and deleting.
 13. Amethod for storing information electronically comprising thesteps:providing a pen and an inputting surface upon which a user draws afreehand figure with the pen; creating a data representation of thefigure; displaying the figure in a main screen area; providing a taggingprocedure for identifying the figure with a tag in response to the usertouching the pen adjacent the figure on one side of the figure, the tagfor the figure being displayed in a subsidiary screen area associatedwith the main screen area; and manipulating the data representation ofthe figure thus tagged in response to user manipulations of the tag. 14.The method of claim 13 wherein a step of selecting a tagged figureprecedes the step of manipulating the data representation of the taggedfigure.
 15. The method of claim 14 wherein the tagging procedure causesa figure thus tagged to also be automatically selected.
 16. The methodof claim 14 wherein the steps of tagging figures, selecting taggedfigures, and manipulating tagged figures are completed only in responseto user inputs in a subsidiary screen area separate from the main screenarea.
 17. The method of claim 16 wherein tagging the figure occurs inresponse to the user tapping the pen in the subsidiary screen areaadjacent the figure, and selecting the figure occurs in response to theuser tapping the pen on the associated tag.
 18. A device for storinginformation electronically comprising:an input surface upon which a usercan indicate a freehand figure with a pen, the device creating a datarepresentation of the figure thus indicated by the user; a main screenarea for displaying the indicated figure; a subsidiary screen area fordisplaying a tag associated with the figure, the tag being created inresponse to a user command indicated by manipulating the pen in an areaadjacent one side of the figure, the user modifying the datarepresentation and display of the figure by pen actions directed to thetag.
 19. The device of claim 18 wherein modification of the datarepresentation and display of the figure includes moving, copying anddeleting the figure.
 20. The device of claim 19 wherein actions directedto the tag for modification of the data representation and display ofthe figure are recognized only in the subsidiary screen area.